Random Forests for Real Time 3D Face Analysis

We present a random forest-based framework for real time head pose estimation from depth images and extend it to localize a set of facial features in 3D. Our algorithm takes a voting approach, where each patch extracted from the depth image can directly cast a vote for the head pose or each of the facial features. Our system proves capable of handling large rotations, partial occlusions, and the noisy depth data acquired using commercial sensors. Moreover, the algorithm works on each frame independently and achieves real time performance without resorting to parallel computations on a GPU. We present extensive experiments on publicly available, challenging datasets and present a new annotated head pose database recorded using a Microsoft Kinec

Real time 3D face alignment with random forests-based active appearance models

Many desirable applications dealing with automatic face analysis rely on robust facial feature localization. While extensive research has been carried out on standard 2D imagery, recent technological advances made the acquisition of 3D data both accurate and affordable, opening new ways to more accurate and robust algorithms. We present a model-based approach to real time face alignment, fitting a 3D model to depth and intensity images of unseen expressive faces. We use random regression forests to drive the fitting in an Active Appearance Model framework. We thoroughly evaluated the proposed approach on publicly available datasets and show how adding the depth channel boosts the robustness and accuracy of the algorithm. © 2013 IEEE.Fanelli G., Dantone M., Van Gool L., ''Real time 3D face alignment with random forests-based active appearance models'', 10th IEEE international conference on automatic face and gesture recognition - FG-2013, 8 pp., April 22-26, 2013, Shanghai, China.status: publishe

Real-time facial feature detection using conditional regression forests

Although facial feature detection from 2D images is a well-studied field, there is a lack of real-time methods that estimate feature points even on low quality images. Here we propose conditional regression forest for this task. While regression forest learn the relations between facial image patches and the location of feature points from the entire set of faces, conditional regression forest learn the relations conditional to global face properties. In our experiments, we use the head pose as a global property and demonstrate that conditional regression forests outperform regression forests for facial feature detection. We have evaluated the method on the challenging Labeled Faces in the Wild [20] database where close-to-human accuracy is achieved while processing images in real-time. © 2012 IEEE.Dantone M., Gall J., Fanelli G., Van Gool L., ''Real-time facial feature detection using conditional regression forests'', 25th IEEE computer society conference on computer vision and pattern recognition - CVPR 2012, pp. 2578-2585, June 18-20, 2012, Providence, Rhode Island, USA.status: publishe

Body Parts Dependent Joint Regressors for Human Pose Estimation in Still Images

In this work, we address the problem of estimating 2d human pose from still images. Articulated body pose estimation is challenging due to the large variation in body poses and appearances of the different body parts. Recent methods that rely on the pictorial structure framework have shown to be very successful in solving this task. They model the body part appearances using discriminatively trained, independent part templates and the spatial relations of the body parts using a tree model. Within such a framework, we address the problem of obtaining better part templates which are able to handle a very high variation in appearance. To this end, we introduce parts dependent body joint regressors which are random forests that operate over two layers. While the first layer acts as an independent body part classifier, the second layer takes the estimated class distributions of the first one into account and is thereby able to predict joint locations by modeling the interdependence and co-occurrence of the parts. This helps to overcome typical ambiguities of tree structures, such as self-similarities of legs and arms. In addition, we introduce a novel data set termed FashionPose that contains over 7,000 images with a challenging variation of body part appearances due to a large variation of dressing styles. In the experiments, we demonstrate that the proposed parts dependent joint regressors outperform independent classifiers or regressors. The method also performs better or similar to the state-of-the-art in terms of accuracy, while running with a couple of frames per second.Dantone M., Gall J., Leistner C., Van Gool L., ''Body parts dependent joint rgressors for human pose estimation in still images'', IEEE transactions on pattern analysis and machine intelligence, vol. 36, no. 11, pp. 2131-2143, November 2014.status: publishe

Discriminative learning of apparel features

© 2015 MVA organization. Fashion is a major segment in e-commerce with growing importance and a steadily increasing number of products. Since manual annotation of apparel items is very tedious, the product databases need to be organized automatically, e.g. by image classification. Common image classification approaches are based on features engineered for general purposes which perform poorly on specific images of apparel. We therefore propose to learn discriminative features based on a small set of annotated images. We experimentally evaluate our method on a dataset with 30,000 images containing apparel items, and compare it to other engineered and learned sets of features. The classification accuracy of our features is significantly superior to designed HOG and SIFT features (43.7% and 16.1% relative improvement, respectively). Our method allows for fast feature extraction and training, is easy to implement and, unlike deep convolutional networks, does not require powerful dedicated hardware.Rothe R., Ristin M., Dantone M., Van Gool L., ''Discriminative learning of apparel features'', 14th IAPR international conference on machine vision applications - MVA 2015, pp. 5-9, May 18-22, 2015, Tokyo, Japan.status: publishe

Augmented faces

In this paper we present a fully automatic system for face augmentation on mobile devices. A user can point his mobile phone to a person and the system recognizes his or her face. A tracking algorithm overlays information about the identified person on the screen, thereby achieving an augmented reality effect. The tracker is running on the mobile client, while the recognition is running on a server. The database on the server is built by a fully autonomous crawling method, which taps social networks. For this work we collected 300 000 images from Facebook. The social context gained during this social network analysis is also used to improve the face recognition. The complete system runs in real time on a state-of-the-art mobile phone and is fully automatic, from offline crawling up to augmentation on the mobile device. It can be used to display more information about the identified persons or as a user interface for mixed reality application. To the best of our knowledge this is the first work, which covers such a system end-to-end. © 2011 IEEE.Dantone M., Bossard L., Quack T., Van Gool L., ''Augmented faces'', 2nd IEEE international workshop on mobile vision - in conjunction with ICCV 2011, November 7, 2011, Barcelona, Spain.status: publishe